Anthropic decision theory for self-locating beliefs 



Stuart Armstrong, 
Future of Humanity Institute, 
Department of Philosophy, 
Oxford University 

2011 



Abstract 

This paper sets out to solve the Sleeping Beauty problem and various 
related anthropic (self-locating belief) problems, not through the calcula- 
tion of anthropic probabilities, but through finding the correct decision to 
make. Given certain simple assumptions, it turns out to be possible to do 
so without knowing the underlying anthropic probabilities. Most common 
anthropic problems are underspecified from the decision perspective, and 
this can explain some of the differing intuitions in the subject: selfless and 
selfish agents, total and average utilitarians, will all reach different deci- 
sions in the same problem. These results are formalised into an anthropic 
decision theory, that is then used to solve many anthropic problems and 
paradoxes, such as the Presumptuous Philosopher, Adam and Eve, and 
Doomsday problems. 
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1 Introduction 

We cannot have been born on a planet unable to support life. This self-evident 
truism is an example of anthropic or self-locating OREF reasoning: we cannot 
see ourselves as 'outside observers' when looking at facts that are connected 
with our own existence. This means that simply by realising that we exist, we 
change the odds of certain things being true or false. 

There have been several different ways of formalising the above intuition. 
The two most popular revolve around the 'Self-Sampling Assumption' and the 
'Self-Indication Assumption' (' |Bos02| ). Both of these assumptions give a way 
of computing anthropic probabilities, answering questions such as 'Given that I 
exist and am human, what probability should I assign to there being billions (or 
trillions) of other humans?'. Unfortunately the two assumptions are incompati- 
ble with each other, and give different answers to standard anthropic problems. 
Nor do they tell you how to translate the probabilities they give you into actual 
decisions: many anthropic problems revolve around identical copies, who will 
presumably decide exactly the same thing as you. In order to figure out the con- 
sequences of your actions, you have to figure out if you bear responsibility for the 
behaviour of identical copies. Causal decision theory [LewSlj would claim that 
you have no such responsibility, while evidential decision theory jGHSlj would 
claim the opposite. And depending on how you resolve this question, SIA and 
SSA may make you always follow the same behaviour, while still proclaiming 
different probabilities! 

Hence this paper does not take any position on anthropic probabilities, and 
instead seeks to find the correct decision in anthropic problems. This seems 
an extraordinary ambition - how can the right decision be made, if the right 
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probabilities aren't known? It turns out that with a few broad assumptions, it 
is indeed possible to solve these problems, and thus get the correct decision. 

This will be illustrated by careful analysis of one of the most famous an- 
thropic problems, the Sleeping Beauty problem |ElgOO| . It turns out that one 
of the reasons that people have had contradictory intuitions on the problem 
is that it is underspecified from the decision perspective. The correct decision 
differs depending on whether the Sleeping Beauties are assumed to be selfless 
or selfish, or total or average utilitarians. 

Many anthropic problems can be similarly solved using these assumptions, 
and this is often enough. But it still leaves many questions unanswered, espe- 
cially those involving non-identical agents. Thus this paper then presents an 
Anthropic Decision Theory, which gives the correct solution to all the Sleep- 
ing Beauty variants and generalises to other anthropic problems. The last part 
of the paper illustrates the application of anthropic decision theory to many 
famous anthropic problems and paradoxes. It provides a more intuitive resolu- 
tion of the Presumptuous Philosopher, Adam and Eve |Bos01) , and Doomsday 
problems |CM83) . 

2 The Sleeping Beauty Problem 

The Sleeping Beauty |ElgOO| problem is one of the most fundamental in an- 
thropic probability, and any decent theory of anthropics needs to be able to 
deal with it. It is related to many similar problems, such as the absent-minded 
driver |AHP96) , the Sailor's Child Problem |Nea06| , the incubator and the pre- 
sumptuous philosopher jBos02l . From the perspective of this paper, all those 
problems are actually the same one, the only difference being in the selflessness 
of the agents. 

In the standard setup. Sleeping Beauty is put to sleep on Sunday, and awoken 
again Monday morning, without being told what day it is. She is put to sleep 
again at the end of the day. A fair coin was tossed before the experiment began. 
If that coin showed heads, she is never reawakened. If the coin showed tails, she 
is fed a one-day amnesia potion (so that she does not remember being awake on 
Monday) and is reawakened on Tuesday, again without being told what day it 
is. At the end of Tuesday, she is left to sleep for ever (see Figure [T]). 




Monday Tuesday 



Figure 1: The Sleeping Beauty problem 
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The incubator variant of the problem jBos02| has no initial Sleeping Beauty, 
just one or two copies of her created (in different, identical rooms), depending 
on the result of the coin flip (see Figure [2]). As we shall see in Section 3.1.2 



the fact there is no initial Sleeping Beauty means that extra assumptions will 
be needed to solve this incubator variant. 




Room 1 Room 2 



Figure 2: The incubator variant of the Sleeping Beauty problem 

The question then is what probability a recently awoken or created Sleeping 
Beauty should give to the coin falling heads or tails and it being Monday or 
Tuesday when she is awakened (or whether she is in Room 1 or 2). There 
are two main schools of thought on the subject, making use of either the Self- 
Sampling Assumption, or the Self-Indication Assumption |Bos02j . 

2.1 The Self-Sampling Assumption 

The self-sampling assumption (SSA) relies on the insight that Sleeping Beauty, 
before being put to sleep on Sunday, expects that she will be awakened in future. 
Thus her awakening grants her no extra information, and she should continue 
to give the same credence to the coin flip being heads as she did before, namely 
1/2. 

In the case where the coin is tails, there will be two copies of Sleeping 
Beauty, one on Monday and one on Tuesday, and she will not be able to tell, 
upon awakening, which copy she is. She should assume that both are equally 
likely. This leads to SSA: 

Definition 2.1 (Self-Sampling Assumption). All other things being equal, an 
observer should reason as if they are randomly selected from the set of all actu- 
ally existent observers (past, present and future) in their reference class. 

There are some issues with the concept of 'reference class' |Bos02| . but here 
it is enough to set the reference class to be the set of all other Sleeping Beauties 
woken up in the experiment. 



4 



Given this, the probabihty calculations become straightforward: 



PssA (Heads) 
PssA (Tails) 



1/2, 
1/2, 
1, 
0, 
1/2, 
1/2. 



PsSA (Monday I Heads) 
-Pss A (Tuesday I Heads ) 
PsSA (Monday I Tails) 
PssA (Tuesday! Tails) 



By Bayes' theorem, these imply that: 



Pss A (Monday) 
Pss A (Tuesday) 



3/4, 
1/4. 



2.2 The Self-Indication Assumption 

There is another common way of doing anthropic probability, namely to use 
the self- indication assumption (SIA). This derives from the insight that being 
woken up on Monday after a heads, being woken up on Monday after a tails, 
and being woken up on Tuesday are all subjectively indistinguishable events, 
which each have a probability 1/2 of happening, therefore we should consider 
them equally probable. This is formalised as: 

Definition 2.2 (Self-Indication Assumption). All other things being equal, an 
observer should reason as if they are randomly selected from the set of all pos- 
sible observers. 

Note that this definition of SIA is slightly different from that used in Bostrom 
[Bos02| : what we would call SIA he designated as the combined SIA-I-SSA. We 
shall stick with the definition above, however, as it is coming into general use. 
Note that there is no mention of reference classes, as one of the great advantages 
of SIA is that any reference class will do, as long as it contains the observers in 
question. 

Given SIA, the three following observer situations are equiprobable (each has 
an 'objective' probability 1/2 of happening), and hence SIA gives them equal 
probabihties 1/3: 



SIA and SSA are sometimes referred to as the thirder and halfer positions re- 
spectively, referring to the probability they give for Heads. 



PsiA (Monday n Heads) 
PsiA(Mondayn Tails) 
PsiA(Tuesdayn Tails) 



1/3, 
1/3, 
1/3. 



This allows us to compute the probabilities: 



PsiA (Monday) 
PsiA (Tuesday) 
PsiA (Heads) 
PsiA (Tails) 



2/3, 
1/3, 
1/3, 
2/3. 
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2.3 Long-run frequencies 

It might be tempting to resolve the dispute between SIA and SSA by appeahng 
to long-run frequencies. But it turns out the criteria for counting these long-run 
frequencies completely determines which assumption comes out on top. 

This is easiest to see in the incubator variant of the Sleeping Beauty problem. 
Imagining running the incubator variant as a million separate experiments, each 
dependent on a separate coin flip. Each created copy is then asked to guess 
whether their coin flip came up heads or tails. If the winning criteria is how 
many of the copies guess right in total across all experiments, then SIA comes 
out ahead: the copies will all guess tails, and 2/3 of them will be right. But 
if instead each experiment simply reported 'all copies guessed right/all copies 
guessed wrong', then guessing heads or tails will give the same score: SSA gives 
the correct odds for this. 

These two different ways of aggregating long-run frequencies are closely re- 
lated to the difference between the selfish, selfless and total and average utili- 
tarian variants discussed in the next section. 

3 Solving Sleeping Beauty 

The probabilities in anthropic situations depend on whether one accepts SSA 
or SIA. However, when confronted with anthropic decision problems, it is often 
possible to determine the correct decision, without reference to either assump- 
tion. To do so, one needs to make use of a few hopefully self-evident axioms. 

First, in order to transform the Sleeping Beauty problem into a decision 
problem, assume that every time she is awoken, she is offered a coupon that 
pays out £1 if the coin fell tails. She must then decide at what cost she is 
willing to buy that coupon. 

The very first axiom is that of temporal consistency. If your preferences 
are going to predictably change, then someone will be able to exploit this, 
by selling you something now that they will buy back for more later, or vice 
versa. This axiom is implicit in the independence axiom in the von Neumann- 
Morgenstern axioms of expected utility |vNMj , where non-independent decisions 
show inconsistency after partially resolving one of the lotteries. It appears, 
under the name diachronic consistency, in David Wallace's work proving the 
Born rule in quantum mechanics. For our purposes, we will define it as: 

Axiom 1 (Temporal consistency). // an agent at two different times has the 
same knowledge and preferences, then the past version will never give up any- 
thing of value in order to change the conditional decision of the future version. 

Here 'conditional decision' means the decision, conditional on extra infor- 
mation the future copy may have. Hence you wouldn't want to determine your 
future copy's decision if they had seen a coin flip that you hadn't; but you would 
want to say 'if they've seen heads, then they should do this, but if they've seen 
tails, then they should do that'. 

This axiom is appropriate for the standard Sleeping Beauty problem, but 
not for the incubator variant, where the different copies are not future or past 
versions of each other. To deal with that, we extend the axiom to: 
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Axiom 1 (Consistency). // two copies of an agent have the same knowledge 
and preferences, then the one version will never give up anything of value in 
order to change the conditional decision of the other version. 

Note that while 'same preferences' is something we could expect to see for 
the same agent at different times, it is not something the case for copies, who 
are generally assumed to be selfish towards each other. Indeed, this whole issue 
of selflessness, altruism and selfishness will be of great import for the agent's 
behaviour, as we shall now see. 

3.1 SIA-like situations 

3.1.1 Selfless Sleeping Beauty 

In order to specify the decision problem fully, we decree that Sleeping Beauty 
has an entirely selfless utility function. It need not be a nice utility function 
- she may be entirely devoted to hugging sick orphans, or her life's goal may 
be drowning cute kittens - but she is entirely selflessly committed to it, above 
any thoughts of herself. To simplify matters, we will further assume her utility 
is linear in cash (though cash for her is simply a tool to accomplish her selfless 
goals) . 

Before Sleeping Beauty is put to sleep the flrst time, she will follow the 
following reasoning: 

"In the tails world, future copies of myself will be offered the same 
deal twice. Any profit they make will be dedicated to hugging or- 
phans/drowning kittens, so from my perspective, profits (and losses) 
will be doubled in the tails world. If my future copies will buy the 
coupon for £x, there would be an expected £0.5(2 x (— x + 1) + 1 x 
{—X + 0)) = £{1 — 2>/2x) going towards my goal. Hence I would 
want my copies to buy whenever x < 2/3." 

Then by the temporal consistency axiom, this is indeed what her future copies 
will do. Note that Sleeping Beauty is here showing a behaviour similar to the 
SIA probabilities - she is betting on 2 : 1 odds that she is in the world with two 
copies. 

3.1.2 Selfless Sleeping Beauty without the initial Beauty 

In some variants of the Sleeping Beauty problem (such as the sailor's child and 
the incubator), there is no initial Sleeping Beauty to make decisions for her 
future copies. Thus consistency is not enough to resolve the decision problem, 
even for a selfless Sleeping Beauty. To do so, we will need to make use of our 
second axiom: 

Axiom 2 (Outside agent). If there exists a collection of identical agents (which 
may he the same agent at different times) with same preferences, then another 
copy of them with the same information would never give up anything of value 
to make them change their conditional decisions. 

With this axiom, the situation reduces to the above selfless Sleeping Beauty, 
by simply adding in the initial Sleeping Beauty again as 'another copy'. 
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Some might feel that that axiom is too strong, that invariance under the cre- 
ation or destruction of extra copies is something that cannot be simply assumed 
in anthropic reasoning. An equivalent axiom could be: 

Axiom 2 (Total agent). If there exists a collection of identical agents (which 
may be the same agent at different times) with same preferences, then they will 
make their decisions as if they were a single agent simultaneously controlling all 
their (linked) conditional decisions. 

This axiom is equivalent to the other, with the total agent taking the role of 
the outside agent. Since all the agents are identical, going through exactly the 
same reasoning process to reach the same decision, the total agent formulation 
may be more intuitive. They are, from a certain perspective, literally the same 
agent. This is the decision that the agents would reach if they could all coor- 
dinate with each other, if there were a way of doing this without them figuring 
out how many of them there were. 



3.2 Altruistic total utilitarian Sleeping Beauty 

Selfless agents (who follow non-personal preferences that don't include any other 
agents' personal preferences) can be distinguished from altruistic agents (who 
follow the aggregate of theirs and other agents' personal preferences). The 
selfless agent only cares about the monetary aspect of a bet in as much as the 
money can be used to further their pet outcome; the altruist cares about the 
monetary aspect of a bet in as much as the money provides personal utility to 
each winner. 

However, an altruistic total utilitarian will have the same preferences over 
the possible outcomes in a Sleeping Beauty situation: the outcomes in the tails 
world is doubled, as any gain/loss happens twice, and the altruist adds up the 
effect of each gain/loss. Hence the altruistic total utilitarian Sleeping Beauty 
will make the same decisions as the selfless one. 



3.2.1 Copy-altruistic total utilitarian Sleeping Beauty 

The above argument does not require that Sleeping Beauty be entirely altruistic, 
only that she be altruistic towards all her copies. Thus she may have selfish 
personal preferences ("I prefer to have this chocolate bar, rather than letting 
Snow White get it"), as long as these are not towards her copies ("I'm indifferent 
as to whether I or Sleeping Beauty II gets the chocolate bar"). And then she 
will make the same decision in this problem as if she was entirely altruistic. 



3.3 SSA-like situations 

3.3.1 Altruistic average utilitarian Sleeping Beauty 



In Section 2.3 we mentioned how different way of combining long run results 
(total correct versus average correct) led to SSA or SIA being considered correct. 
Similarly, average and total utilitarians will behave differently in the Sleeping 
Beauty problem. 

In the incubator variant, consider the reasoning of an Outside/Total agent 
(as in axiom [2]): 
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"If the various Sleeping Beauties decide to pay £x for the coupon, 
they will make — £x in the heads world. In the tails world, they will 
each make £(\ — x) each, so an average of £(1 — x). This give me 
an expected utihty of £0.5(— a; + (1 — x)) ~ £(0.5 — x), so I would 
want them to buy the coupon for any price less than £0.5." 

And this will then be the behaviour the agents will follow. Thus they would be 
behaving as if they were following SSA odds, and putting equal probability on 
the heads versus tails world. 

For a version of this that makes senses for the classical Sleeping Beauty 
problem, one could imagine that she to be awakened a week after the experiment. 
Further imagine she would take her winnings and losses during the experiment 
in the form of chocolate, consumed immediately. Then because of the amnesia 
drug, she would only remember one instance of this in the tails world. Hence if 
she valued memory of pleasure, she would want to be average utilitarian towards 
the pleasures of her different versions, and would follow SSA odds. 

3.3.2 Reference classes and copy-altruistic agents 

Standard SSA has a problem with reference classes. For instance, the larger the 
reference class becomes, the more the results of SSA in small situations become 
similar to SIA. The above setup mimics the effect: if there is a very large 
population of outsider individuals that Sleeping Beauty is altruistic towards, 
then the gains to two extra copies will tend to add, rather than average: if f2 is 
large, then 2a:/ (2 + fi) (averaged gain to two created agents each gaining x) is 
approximately twice x/(l + fi) (averaged gain to one created agent gaining x), 
so she will behave more closely to SIA odds. 

This issue is not present for copy- altruistic average utilitarian Sleeping Beau- 
ties, as she doesn't care about any outsiders. 

3.3.3 Selfish Sleeping Beauty 

In all of the above example, the goals of one Sleeping Beauty were always in 
accordance with the goals of her copies or the past and future versions of herself. 
But what happens when this fails? What happens when the different versions 
are entirely selfish towards each other? Very easy to understand in the incuba- 
tor variant (the different created copies feel no mutual loyalty), it can also be 
understood in the standard Sleeping Beauty problem if she is a hedonist with a 
high discount rates. 

Since the different copies do have different goals, the consistency axioms 
no longer apply. It seems that we cannot decide what the correct decision 
is in this case. There is, however, a tantalising similarity between this case 
and the altruistic average utilitarian Sleeping Beauty. The setups (including 
probabilities) are the same. By 'setup' we mean the different worlds, their 
probabilities, the number of agents in each world, and the decisions faced by 
these agents. Similarly, the possible 'linked' decisions are the same. See Section 
|4.2| for a proper definition of linked decisions; here it just means that all copies 
will have to make the same decision, so there is one universal 'buy coupon' or 
'reject coupon'. And, given this linking, the utilities derived by the agents is 
the same for either outcome in the two cases. 
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To sec this, consider the selfish situation. Each Sleeping Beauty will make 
a single decision, whether to buy the coupon at the price offered. Not buying 
the coupon nets her £0 in all worlds. Buying the coupon at price £x nets her 
— £x in the heads world, and £{1 — x) in the tails world. The linking is present 
but has no impact on these selfish agents: they don't care what the other copies 
decide. 

This is exactly the same for the altruistic average utilitarian Sleeping Beau- 
ties. In the heads world, buying the coupon at price £x nets her —£x worth 
of utility. In the tails world, it would net the current copy £{1 — x) worth of 
individual utility. Since the copies arc identical (linked decision), this would 
happen twice in the tails world, but since she only cares about the average, this 
grants both copies only £{l — x) worth of utility in total. The linking is present, 
and has an impact, but that impact is divided by the average utilitarianism of 
the copies. 

Thus the two situations have the same setup, the same possible linked deci- 
sions and the same utility outcomes for each possible linked decision. It would 
seem there is nothing relevant to decision theory that distinguishes these two 
cases. This gives us the last axiom: 

Axiom 3 (Isomorphic decisions). // two situations have the same setup, the 
same possible linked decisions and the same utility outcomes for each possible 
linked decision, and all agents are aware of these facts, then agents should make 
the same decisions in both situations. 

This axiom immediately solves the selfish Sleeping Beauty problem, implying 
that agents there must behave as they do in the Altruistic average utilitarian 
Sleeping Beauty problem, namely paying up to £0.50 for the coupon. In this 
way, the selfish agents also behave as if they were following SSA probabilities, 
and believed that heads and tails were equally likely. 

3.4 Summary of results 

We have broadly four categories of agents, and they follow two different types 
of decisions (SIA-like and SSA- like). In the Sleeping Beauty problem (and in 
more general problems), the categories decompose as: 

1. Selfless agents who will follow SIA-type odds. 

2. (Copy-)Altruistic total utilitarians who will follow SIA-type odds. 

3. (Copy-)Altruistic average utilitarians who will follow SSA-type odds. 

4. Selfish agents who will follow SSA-typo odds. 

For the standard Sleeping Beauty problem, the first three decisions derived from 
consistency. The same result can be established for the incubator variants using 
the Outside/Total agent axioms. The selfish result, however, needs to make use 
of the Isomorphic decisions axiom. 
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4 Anthropic decision theory 



In this section, we will see a decision procedure that is compatible with all the 
axioms above, and applies to more general anthropic and linked decision making 
problems. 

4.1 Probabilities and decisions 

SSA and SIA both give you methods for determining the probabilities in the 
Sleeping Beauty problem, but they don't tell you what decisions to make. Even 
apart from the selfless/selfish distinction above in Section [s] one has to worry 
about the total impact of the decision, when several agents are making the same 
decision. For example, consider a setup where all copies of Sleeping Beauty have 
to vote unanimously on whether to implement a particular policy. Since they 
are identical, they will vote the same way. One could use a principle of divided 
responsibility: when there are two copies, each one is responsible for a half of 
the ultimate decision (this could be termed the causal decision theory |Lew81| 
position). Or one could use a principle of total responsibility: each copy is 
responsible for the totality of the decision since if they voted otherwise, the 
decision would fail (this could be termed the evidential decision theory |GH81| 
position) . 

Note that SIA with divided responsibility would reach the same decisions as 
SSA with total responsibility - the doubling of odds in the heads world exactly 
compensated by the halving of responsibility. In this setting, the probabilities 
themselves become meaningless: the first agent may claim 'SIA!' while the sec- 
ond claim 'SSA!', but they will agree on every decision, making those simply 
slogans. If someone shouts 'Vive la revolution!' while the other shouts 'Vive 
Ic Roil', then we would expect them to sometimes reach different decisions; if 
not, their claims are empty. They may believe that the 'true' probability is 1/3 
or 1/2, but since this never shows up in a difference in action, it is hard to see 
what these beliefs actually mean. 

For this reason this paper takes the actual decisions as the fundamental 
objects of interest, with the probabilities being merely tools, and having no 
separate objective meaning. 

4.2 Linked decisions and Anthropic Decision Theory 

Identical copies of Sleeping Beauty will make the same decision when faced 
with same situations (technically true until quantum and chaotic effects cause 
a divergence between them, but most decision processes will not be sensitive 
to random noise like this). Similarly, Sleeping Beauty and the random man on 
the street will make the same decision when confronted with a twenty pound 
note: they will pick it up. However, while we could say that the first situation is 
linked, the second is coincidental: were Sleeping Beauty to refrain from picking 
up the note, the man on the street would not so refrain, while her copy would. 

The above statement brings up subtle issues of causality and counterfactuals, 
a area of dispute between causal |Lew81| and evidential decision theory |GH81j . 
To sidestep the issue entirely, let us recast the problem in programming terms, 
seeing the agent's decision process as a deterministic algorithm. If agent a is 
an agent that follows an automated decision algorithm then if A knows its 
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own source code (by quining |BM72| for instance), it might have a Hne saying 
something hke: 

Module M: If B is another algorithm, belonging to agent /3, identical 
with A ('yourself'), assume A and B will have identical outputs on 
identical inputs, and base your decision on this. 

This could lead, for example, to a and /? cooperating in a symmetric Prisoner's 
Dilemma. And there is no problem with A 'believing' the above assumption, 
as it is entirely true: identical deterministic algorithms on the same input do 
produce the same outputs. With this in mind, we give an informal definition of 
a linked decision (either conditional or standard) as: 

Definition 4.1 (Linked decisions). Agent a's decisions are linked with agent 
(3 's, if both can prove they will both make the same decision, even after taking 
into account the fact they know they are linked. 

An example of agents that are not linked would be two agents a and /?, 
running identical algorithms A and B on identical data, except that A has 
module M while B doesn't. Then A's module might correctly deduce that they 
will output the same decision, but only if A disregards the difference between 
them, i.e. module M itself. So A can 'know' they will output the same decision, 
but if it acts on that knowledge, it makes it incorrect. If A and B both had 
module M, then they could both act on the knowledge and it would remain 
correct. Generally, identical causal decision theorists [LewSlj will not be linked, 
while identical evidential decision theorists |GH81j will be. 

The 'objective probabilities' of a world are the prior probabilities given (such 
as the 50 — 50 odds in the Sleeping Beauty problem), updated, as usual, by each 
agent applying Bayes theorem to the likelihood of them making the observations 
they have made. Given this definition, anthropic decision theory (ADT) can be 
simply stated as: 

Anthropic Decision Theory (ADT) An agent should first find 
all the decisions linked with their own. Then they should maximise 
expected utility, acting as if they simultaneously controlled the out- 
comes of all linked decisions, and using the objective (non-anthropic) 
probabilities of the various worlds. 

ADT is similar to SSA in that it makes use of reference classes. However, 
SSA needs to have the reference class information established separately before 
it can calculate probabilities, and different reference classes give very different 
results |Bos02j . In contrast, the reference class for ADT is part of the definition. 
It is not the class of identical or similar agents; instead, it is the class of linked 
decisions which (by definition) is the class of decisions that the agent can prove 
are linked. Hence the whole procedure is perfectly deterministic, and known for 
a given agent. 

Now, module M is enough when the agents/algorithms are strictly identical, 
but fails when they differ slightly. For instance, imagine a variant of the selfless 
Sleeping Beauty problem where the two agents aren't exactly identical in tails 
world. There are two separate agents, a and a', where they are both selflessly 
devoted to the same cause, but a' has some personal displeasure in engaging 
in trade - if she buys the coupon, she will suffer a single —£0.05 penalty for 
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doing so. Agent a is indifferent to the other agent suffering this penalty. In the 
tails world, there will be one of each type, while in the heads world, the single 
agent is equally likely to be of either (trade-neutral or trade-disliking) type. 
The objective probabilities are the same as before: any given Sleeping Beauty 
has 50% probability of being agent a in the tails world (because she is one of 
two) or the heads world (because each is equally probable). Thus updating on 
realising she is agent a or a' does not change her priors. 

Then if the coupon is priced at £0.60, something quite interesting happens. 
If the agents do not believe they are linked, they will refuse the offer: their 
expected returns are 0.5(-0.6 -I- (1 - 0.6)) = -0.1 and -0.1 - 0.05 = -0.15 
respectively. If however they believe their decisions are linked, they will calculate 
the expected return from buying the coupon as 0.5(— 0.60 -I- 2(1 — 0.60)) = 0.1 
and 0.1 — 0.05 = 0.05 respectively. Since these are positive, they will buy the 
coupon: meaning their assumption that they were linked was actually correct! 

If the coupon is priced at £0.66, things change. If the two agents assume 
their decisions are linked, then they will calculate their expected return from 
buying the coupon as 0.5(-0.66 -h 2(1 - 0.66)) = 0.01 and 0.01 - 0.05 = -0.04 
respectively. The first agent will buy, and the second will not - they were wrong 
to assume they were linked 

A more general module that gives this kind of behaviour is: 

Module iV: Let H be the hypothesis that the decision of A ('myself') 
and those of algorithm B are linked. I will then compute what 
each of us will decide if we were both to accept H. If our ultimate 
decisions are indeed the same, and if the other agent also has a 
module A^, then I will accept H. 

The module N gives correct behaviour. It only triggers if the agents can prove 
that accepting H will ensure that H is true ~ and then A^ makes them accept 
_ff , hence making H true. 

For the coupon priced at £0.60, it will correctly tell them they are linked, 
and they will both buy it. For the coupon priced at £0.66, it will not trigger, 
and both will refuse to buy it - though they reach the same decision, they will 
not have done so if they had assumed they were linked. For a coupon priced 
above £2/3, module N will correctly tell them are linked again, and they will 
both refuse to buy it. 

4.2.1 Precommitments 

Another important problem in decision theory is Parfit's hitchhiker |Par84| and 
similar situations. Here, a starving hitchhiker meets a driver who could drive 
him to the nearest town. The hitchhiker promises to pay the driver a reward 
upon arrival. Unfortunately, both the driver and hitchhiker are selfish and 
perfectly rational, and realise the hitchhiker will have no incentive to pay once 
he is driven to the town. So the driver motors away, leaving him to die. There is 
a lot of literature on this subject, which will not be summarised here; the main 
point being that it is advantageous for an agent to be able to make binding public 
precommitments. These can be seen as an extension of temporal consistency: 
once an agent has decided what to do, it commits its future copies to the same 
behaviour, to avoid the possibility of predictably losing utility to any agent that 
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can exploit its inconsistency. To allow ADT agents to succeed in situations like 
this, we define 'decisions' to include precommitments. 



4.3 Axioms 

Let's look at the various axioms laid down in Section [Sj and show that ADT 



(as defined in the previous Section 4.2) obeys them all: 



1. Consistency: If two copies of an agent have the same knowledge and pref- 
erences, then the one version will never give up anything of value in order 
to change the conditional decision of the other version. 

2. Total agent: If there exists a collection of identical agents (which may 
be the same agent at different times) with same preferences, then they 
will make their decisions as if they were a single agent simultaneously 
controlling all their (linked) conditional decisions. 

3. Isomorphic decisions: If two situations have the same setup, the same 
possible linked decisions and the same utility outcomes for each possible 
linked decision, and all agents are aware of these facts, then agents should 
make the same decisions in both situations. 

To show that ADT obeys the consistency axiom, merely note that the deci- 
sion of identical ADT agents with same preferences facing the same (conditional) 
decision will be the same, that both agents know this, and hence their decisions 
must be linked. ADT makes each agent act like a total agent, so that axiom 
must be satisfied. Finally the isomorphic decisions algorithm says, in effect, "if 
two situations differ only by information that is not relevant to ADT, then ADT 
will decide the same way in both" . 

Now since ADT obeys all these axioms, it must give the correct decision as 
described in the various Sleeping Beauty variants of Section |3] 

It would be nice if these axioms were enough to define ADT, but that is 
not the case. They all require exact equality between the preferences of the 
agents in various situations, but ADT simply needs the linking of their decisions. 



As we saw in the previous Section 4.2 not perfectly identical agents can still 



coordinate. The important assumption underlying module N was that if the 
agents assumed they were linked, and then proceeded to maximise their utility 
on that assumption, they would indeed end up linked. This would be the case 
for agents that all agreed which was the best option among all the possible 
(linked) decisions. To cover these cases, we can extend the Isomorphic decisions 
axiom to: 

Axiom 3 (Isomorphic best decision). If two situations have the same setup, the 
same possible linked decisions and the same best outcome for all agents among 
the possible linked decisions, and all agents are aware of these facts, then agents 
should make the same decisions in both situations. 

These three axioms then establish ADT: the ADT solution is the one that 
all agents agree is the best linked decision available. Then we can transform 
this into situation where identical agents have identical utilities, but without 
changing the best linked decision, and the first two axioms are then enough to 
show the agents will indeed choose the ADT solution. 
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4.3.1 Probabilistic linking 



Now, in the real world, rather than requiring a mathematical proof of linking, 
it would probably be more useful to talk of the probability of two decisions 
being linked. One might for instance be willing to cooperate with broadly 
similar people in a Prisoner's Dilemma. ADT should be adaptable to handle 
uncertainty over linkings of decisions the same way it handles all uncertainty 
- for instance it can model the selfish Sleeping Beauty problem as a 50 — 50 
mixture of selfless Sleeping Beauty and Sleeping Anti-Beauty (see Section 5.4 1. 
That approach is beyond the scope of the current paper, however. Instead, 
the next section will delve into the applications of ADT to various common 
anthropic problems. 



5 Applying Anthropic Decision Theory 
5.1 The Presumptuous Philosopher 

The Presumptuous Philosopher was introduced by Nick Bostrom |Bos02| as a 
way of pointing out the absurdities in SIA. In the setup, the universe either has a 
trillion observers, or a trillion trillion trillion observers, and physics is indifferent 
as to which one is correct. Some physicists are preparing to do an experiment 
to determine the correct universe, until a presumptuous philosopher runs up to 
them, claiming that his SIA probability makes the larger one nearly certainly 
the correct one. In fact, he will accept bets at a trillion trillion to one odds 
that he is in the larger universe, repeatedly defying even strong experimental 
evidence with his SIA probability correction. 

What does ADT have to say about this problem? Implicitly, when the prob- 
lem is discussed, the philosopher is understood to be selfish towards any putative 
other copies of himself (similarly. Sleeping Beauty is often implicitly assumed 
to be selfless, which may explain the divergence of intuitions that people have 
on the two problems). Are there necessarily other similar copies? Well, in or- 
der to use SIA, the philosopher must believe that there is nothing blocking the 
creation of presumptuous philosophers in the larger universe; for if there was, 
the odds would shift away from the larger universe (in the extreme case when 
only one presumptuous philosopher is allowed in any universe, SIA finds them 
equiprobable) . So the expected number of presumptuous philosophers in the 
larger universe is a trillion trillion times greater than the expected number in 
the small universe. 

Now if the philosopher is indeed selfish towards his copies, then ADT reduces 
to SSA-type behaviour: the philosopher will correctly deduce that in the larger 
universe, the other trillion trillion philosophers or so will have their decision 
linked with his. However, he doesn't care about them: any benefit that accrue 
to them are not of his concern, and so if he correctly guesses that he resides 
in the larger universe, he will accrue a single benefit. Hence there will be no 
paradox: he will bet at 1 : 1 odds of residing in either the larger or the smaller 
universe. 

If the philosopher is an altruistic total utilitarian, on the other hand, he 
will accept bets at odds of a trillion trillion to one of residing in the larger 
universe. But this no longer counter-intuitive (or at least, no more counter- 
intuitive than maximising expect utility with very small probabilities): the other 
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presumptuous philosophers will make the same bet, so in the larger universe, 
their total profit and loss will be multiplied by a trillion trillion. And since the 
philosopher is altruistic, the impact on his own utility is multiplied by a trillion 
trillion in the large universe, making his bets rational. 

At this point, it might be fair to ask what would happen if some of the 
philosophers were altruistic while others were selfish. How would the two in- 
teract; would the selfless philosopher incorrectly believe his own decision was 
somehow 'benefiting' the selfish ones? Not at all. The decisions of the selfless 
and selfish philosophers are not linked: they both use ADT, but because they 
have very different utilities, they cannot prove that their decisions are linked. 
Which is fortunate, because they aren't. 

5.2 Adam and Eve 

The presumptuous philosopher thought experiment was designed to show up 
problems with SIA reasoning. Another thought experiment by the same author 
[BosOl] was designed to show problems with SSA reasoning. 

In this thought experiment, Adam and Eve, the first two humans, have the 
opportunity to breed or not to breed. If they breed, they will produce trillions of 
trillions of descendants (see Figure[3|. Under SSA odds, the probability of being 
Adam or Eve in a universe with trillions of trillion humans is tiny, while the 
corresponding probability in a universe with just two observers is one. Therefore 
Adam and Eve should conclude that whatever they do, it is tremendously un- 
likely that they will succeed in breeding. Nick Bostrom them proceeds to draw 
many amusing consequences from this 'future affecting' paradox, such as the 
couple forming the firm intention of having sex (and hence risking pregnancy) if 
an edible animal doesn't wander through the entrance of their cave in the next 
few minutes. There seems to be something very wrong with the reasoning, but 
it is a quite natural consequence of SSA. 

What does ADT have to say on the problem? This depends on whether 
the decisions of Adam and Eve and their (potential) descendants arc linked. 
There is prima facie no reason for this to be the case; in any case, how could 
potential future descendants make the decision as to whether Adam and Eve 
should breed? One way of imagining this is if each human is born fully rational, 
aware of the possible world they is born into, but in ignorance as to their identity 
and position in that world. Call this the ignorant rational baby stage. They can 
then make a decision as to what they would do, conditional upon discovering 
their identity. They may then decide or not to stick to their decision. Hence we 
can distinguish several scenarios: 

1. These agents have no 'ignorant rational baby stage', and do not take it 
into account. 

2. These agents have an 'ignorant rational baby stage', but do not allow 
precommitments. 

3. These agents have an 'ignorant rational baby stage', but do allow precom- 
mitments. 

4. These agents have no 'ignorant rational baby stage', but require them- 
selves to follow the hypothetical precommitments they would have made 
had they had such a stage. 
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Figure 3: The Adam and Eve problem: what are the odds of descendants? 



To make this into a decision problem, assume all agents are selfish, and know 
they will be confronted by a choice between a coupon Ci that pays out £1 to 
Adam and Eve if they have no descendants and C2 that pays out £1 to Adam 
and Eve if they have (trillions of trillions of) descendants. Assume Adam and 
Eve will have sex exactly once, and the (objective) chance of them having a 
successful pregnancy is 50%. Now each agent must decide on the relative values 
of the two coupons. 

Obviously in situation [l] the decisions of Adam and Eve and their descen- 
dants are not linked, and ADT means that Adam and Eve will value Ci com- 
pared with C2 according to their objective estimates of having descendants, 
i.e. they will value then equally. There is no SSA-like paradox here. Their even- 
tual descendants will also value the coupons as equally worthless, as they will 
never derive any value from them. 

Now assume there is an 'ignorant rational baby stage'. During this stage, 
the decisions of all agents are linked, as they have the same information, the 
same (selfish) preferences, and they all know this. Each rational baby can then 
reason: 

"If I am in the world with no descendants, then I am Adam or 
Eve, and the Ci coupon is worth £1 to me (and C2 is worthless). 
If, on the other hand, I am in the world with trillions of trillions 
of descendants, there is only two chances in 2 -I- 10^'* of me being 
Adam or Eve, so I value the C2 coupon at £2/(2 + 10^^) (and Ci is 
worthless). These worlds are equiprobable. So I would value Ci as 
being 1 -I- 0.5 • 10^^ times more valuable than C2." 
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So the rational babies in situations [2] and [3] would take Ci as much more valu- 
able than C2, if the deal was proposed to them immediately. Since there are 
no precommitments in situation [2] once the rational babies discover who they 
actually are, they would revert to situation [T] and take them as equally valuable. 
If precommitments are allowed, then the rational babies would further reason: 

"The strategy 'upon discovering I am Adam or Eve, take Ci' nets 
me an expected i^l/2, while the strategy 'upon discovering I am 
Adam or Eve, take C2' nets me an expected i^l/(2 + 10^^), because 
it is very unlikely that I would actually discover that I am Adam 
and Eve. Hence the strategy 'upon discovering I am Adam or Eve, 
accept trades between Ci and C2 at 2 : (2 + 10^^) ratios' is neutral 
in expected utility, and so I will now precommit to accepting any 
trade at any ratios slightly better than this." 

So in situation [Sj even after discovering that they are Adam or Eve, they will 
continue to accept deals at ratios that would seem to imply that they believe in 
the SSA odds, i.e. that they are nearly certain to not have descendants. But it 
is a lot less of a paradox now; it simply arises because there was a time when 
they were uncertain as to what their actual position was, and the effects of this 
uncertainty were 'locked in' by their precommitment. 

Situation [4] is very interesting. By construction, it reproduces the seemingly 
paradoxical behaviour, but here there was never a rational baby stage where the 
behaviour made sense. Why would any agent follow such a behaviour? Well, 
mainly because it allows trade between agents who might not otherwise be able 
to agree on a 'fair' distribution of goods. If all agents agree to the division that 
they would have wanted had they been ignorant of their identity (a 'rawlsian 
veil of ignorance' situation [Raw71^), then they can trade between each other 
without threats or bargaining in these simple cases. 

If the agents are simply altruistic average utilitarians, then Adam and Eve 
would accept SSA odds in all four situations; things that benefit them specif- 
ically are weighted more highly in a universe with few people. So the varying 
behaviour above is a feature of selfishness, not of SSA-type behaviour, and it 
seems precommitments become very important in the selfish case. This certainly 
merits further study. Temporal consistency is a virtue, but does it extend to 
situations like this, where the agent makes binding decisions before knowing 
their identity? Certainly, if the agent had to make a decision immediately, and 
if there were anyone around to profit from temporal inconsistencies, the agent 
should remain consistent, which means following precommitments. However 
this is not entirely obvious that it should still be the case if there were no-one 
to exploit the inconsistency. 

This is not, incidentally, a problem only of ADT - SIA has similar problem 
under the creation of 'irrelevant' selfless agents who don't yet know who they 
are, while SSA has problems under the creation of agents who don't yet know 
what reference class they are in. 

5.3 The Doomsday argument 

Closely related to the Adam and Eve paradox, though discovered first, is the 
Doomsday argument |CM83| . Based on SSA's preference for 'smaller' universes, 
it implies that there is a high probability of the human race becoming extinct 
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within a few generations - at the very least, a much higher probabihty than 
objective factors would imply. 

Under SIA, the argument goes away f |Die07p . so it would seem that ADT 
must behave oddly: depending on the selfishness and selflessness of the agents, 
they would give different probabilities to the extinction of the human race. 
This is not the case, however. Recall that under ADT, decisions matter, not 
probabilities. And agents that are selfish or average utilitarians would not be 
directly concerned with the extinction of the human race, so would not act in 
bets to prevent this. 

This is not a specious point - there are ways of constructing the doomsday 
argument in ADT, but they all rely on odd agents who are selfish with respect to 
their own generation but selfless with respect to the future survival of the human 
race. This lacks the potency of the original formulation: having somewhat odd 
agents behaving in a somewhat odd fashion is not very surprising. For the 
moment, until a better version is produced, we should simply say that the 
doomsday argument is not present in ADT. 

5.4 Sleeping Anti-Beauty 

Sleeping Anti-Beauty is a thought experiment similar to the Sleeping Beauty 
experiment, but with one important caveat: the two copies in the tails world 
hate each other. This works best if the two copies are duplicates, rather than 
the same person at different times. One could imagine, for instance, that a week 
after the experiment, all copies of Sleeping Beauty are awakened and made to 
fight to the death - maybe they live in a civilization that prohibits more than 
one copy of an agent from existing. The single copy in the heads world will be 
left unmolested, as she has nobody to fight. 

That means that the Sleeping Beauties in the tail world are in a zero sum 
game; any gain for one is a loss for the other, and vice-versa. Actually, we need 
a few more assumptions for this to be true: the Sleeping Beauties have to be 
entirely selfish apart from their rivalry, and they do not get offered any goods 
for immediate consumption. Given all these assumptions, what does ADT have 
to say about their decisions? 

As in Section [3) all existent copies of Sleeping Beauty are offered a coupon 
that pays out £1 if the coin fell tails, and asked how much she would be willing 
to give for that. ADT reasoning proceeds for each agent as follows: 

"In the heads world, if I pay £x, all that happens is that I lose £x. 
In the tails, world, if I pay £x, I gain £{1 — x). However my other 
hated copy will make the same decision, and also gain £(l — x). This 
causes me the same amount of loss as the gain of £{1 — x) does, so I 
gain nothing at all in the tails world, whatever £x is. So I value the 
coupon precisely at zero: I would not pay any amount to get it." 

In this, and other similar decisions. Sleeping Beauty would act as if she had an 
absolute certainty of being in the heads world, offering infinity to one odds of 
this being the she cannot realise any gains - or losses! - in the tails 

world. 

It should be noted that selfish Sleeping Beauty can be correctly modelled by 
seeing it as a 50 — 50 mix of selfless Sleeping Beauty and Sleeping Anti-Beauty. 
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6 Conclusion 



Anthropic decision theory is a new way of dealing with anthropic (self locating 
belief) problems, focused exclusively on finding the correct decision to make, 
rather than the correct probabilities to assign. It deals successfully with many 
classical anthropic puzzles, and resolves various paradoxes such as the Presump- 
tuous Philosopher and the Adam and Eve problem, in ways that don't clash with 
intuition. 

In many situations, ADT is moreover a consequence of simple axioms of con- 
sistency, total agents and isomorphism. In these problems, standard intuitions 
were all over the place, partially because people were not clearly distinguishing 
between probabilities and decisions. This lead to the problem being underspeci- 
fied: since people were looking for probabilities, facts such as whether the agents 
were selfless or selfish, total or average utilitarians, were not stated and assumed 
not to matter - though they were often used in the implicit intuitive defence 
of various positions. But in making a decision, they matter tremendously: the 
difference between SIA-like and SSA-like behaviour is down to how copies value 
each other's goals. 

References 

[AHP96] Robert J. Aumann, Sergiu Hart, and Motty Perry. Absent-minded 

driver. Proceedings of the 6th conference on Theoretical aspects of 

rationality and knowledge, 1996. 

[BM72] Paul Bratley and Jean Millo. Computer recreations: Self-reproducing 
automata. Software Practice and Experience, 2:397-400, 1972. 

[BosOl] Nick Bostrom. The doomsday argument, adam & eve, un-|— h, and 
quantum joe. Synthese, 127(3), 2001. 

[Bos02] Nick Bostrom. Anthropic Bias: Observation Selection Effects in Sci- 
ence and Philosophy. Routledge: New York, 2002. 

[CM83] Brandon Carter and W. H. AfcCrea. The anthropic principle and its 
implications for biological evolution. Philosophical Transactions of the 
Royal Society of London, 310(1512):347-363, 1983. 

[Die07] Dennis Dieks. Reasoning about the future: Doom and beauty. Syn- 
these, 156(3), 2007. 

[ElgOO] A. Elga. Self- locating belief and the sleeping beauty problem. Analysis, 
60:143-147, 2000. 

[GH81] A. Gibbard and W. L. Harper. Counterfactuals and two kinds of 
expected utility. Ifs: Conditionals, Beliefs, Decision, Chance, and 
Time, pages 153-190, 1981. 

[Lew81] D. Lewis. Causal decision theory. Australasian Journal of Philosophy, 
59(l):5-30, 1981. 

[Nea06] R. M. Ncal. Puzzles of anthropic reasoning resolved using full non- 
indexical conditioning. 2006. Arxiv preprint math/0608592. 



20 



[Par84] Derek Parfit. Reasons and Persons. Clarendon Press, Oxford, 1984. 

[Raw71] JohnRawls. A Theory of Justice. Cambridge, Massachusetts: Belknap 
Press, 1971. 

[vNM] John von Neumann and Oskar Morgenstern. Theory of Games and 
Economic Behavior. 



21 



